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ABSTRACT 

State colleges and universities (SUs) are r>eing 
forced by critics of higher education to reassess themselves and 
redirect their efforts. There is a movement to discover a simplistic 
and valid means of measuring excellence in higher education. 
Measuring student outcomes cuts through all the difficulties created 
by more standard attempts to find a relieUDle indicator of the 
educational effectiveness of institutions. However, the organization, 
funding, objectives, and directions of many SUs are ill-suited to the 
outcomes movement, and they react with hostility to any attempt by a 
state board or accrediting agency to measure their worth on a 
one-dimensional scale (since it seems unfair to make "what students 
learn" the basis for judging institutional quality and 
effectiveness) . The future may bring a compromise in which outcomes 
measurement becomes an important factor in the financial formula by 
which SUs are supported. An examination of the reasons behind the 
University of Louisville's reluctance to embrace the outcomes 
movement is presented to shed some light on the way other SUs might 
react to such a situation. Some of the suggestions for narrowing the 
gap between advocates of student outcomes aoS**<5sment and those 
reluctant to adopt it are: successful measurement requires an 
atmosphere of openness and informality; multiple measures of student 
Arturatinn innst us^d* and outcojp.es can bs msssurcd in cill of the 
activities of an institution. It is noted that improving education is 
unlikely to be the road to fame for any institution. (SM) 
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Stephen D. Spangehl 

The University of Louisville 
Lou1 SV1 lie, Kentucky 
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made to the Colleye Board National Forum, 
October 26, 1986, The Waldorf-Astoria, 
New York, New York 

State colleges and universities (henceforth SU's, for convenience) are 

beginning to find themselves pulled by two powerful forces they often perceive 

as contradictory. Accrediting agencies, state governing and coordinating 

boards, and public opinion (expressed through a series of reports on higher 

education) are joining together to proclaim that American higher education 

needs to be refocused, needs to concentrate its efforts and resources on what 

many--inc1uding the majority of taxpayers on whom, ultimately, the SU's depend 

for their support--say should be their primary task: the education of 

students, particularly undergraduates. The arguments supportinr? this crusade 

are well known: the skills of college graduates are poorer than ever before, 

and still dropping; the baccalaureate degree has lost its distinction; SU's 

are Inefficient, forever asking for more of a state's resources but unwilling 

to demonstrate that they can use the resources they already have effectively; 

(\ primary and secondary school teachers, whose education is one of the SU's 

^ mS? ' «^ reproduce: this 

•vi^icMiAL HAS BEEN GRANTiZD BY ^* oepaittment or education 

OHtce of Educational Reaenrch and Improvement 

Stephen D, educational resouhces information 



CENTER (ERIC) 
<0Thi$ document has baen reproduced aa 
rac«i/ad from the parson or organization 
Of jinatir>g it 
□ Minor Changes have baen made to improve 



TC THE EDUCATIONAL RESOURCES reproducti on qua i>ty^ 



^ O '^"^CHMATION center (ERIC) • Pomts of viaw or opinions stated m this docu 

CD!/^ '"-nn^/. rp,nt do not nacawanly feP'ewnt official 

• ^ OERlpO»'t»OnorpOhCV 



SPANGEHL - 2 



primary responsibilities, are less well-prepared than they should be; and so 
on. Many, perhaps most, of the criticisms leveled against higher education by 
reports like Involvement Tn Learning are true, at least In part, and something 
clearly needs to be done to get our institutions back on the right track. 
SU's, sensitive to political pressure and dependent upon public opinion 
(expressed through state legislatures) are an easy target for such criticism; 
surely they, through a combination of pressure, legislation, and financial 
reward (or penalty), can be redirected to the purpose for which they are 
funded: educating the children of taxpayers. 

The critics of higher education have, In addition, hit upon an attractive 
mechanism to force the SU's to reassess themselves and redirect their 
efforts. Measuring student outcomes—what students *^ave learned as a result 
of their entire college experience—cuts through all the difficulties created 
by more traditional attempts to find a reliable indicator of the educational 
effectiveness of Institutions. The more commonplace measures of quail ty--the 
resources available to an Institution, the rating of faculty proficiency as 
teachers, the qualifications of faculty, student-faculty ratios, etc. —are all 
controversial to one degree or another. Trying to evaluate educational 
quality by examining the "Inputs" to the system and the "processes" used has 
been unconvincing; focusing on the product?- -the "outputs" or "outcomes" — 
seems to promise a methodology for making vtlld comparisons between 
institutions that otherwise have little In common. If College A consistently 
produces better educated students than CoTege B, a fact that can be 
documented with test scores and other objective measures^ then differences in 
resources, proportion of Ph.D.'s, volumes in the library, educational 
philosophy, grading scales, and other measiras matter little: A is a more 
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effective educational institution than B. We live, after all, in a society 
consistently concerned with "the bottom line," not with all the minute figures 
which are added up to get "the bottom line." Why should we evaluate higher 
education differently? 

Pitted against this movement to discover a simplistic and valid means of 
measuring excellence in higher education are a host of SU's which have 
traditionally seen their missions as more complex than simply turning out 
students who can outperform the graduates of other schools on tests. The 
organization, funding, objectives, and directions of many SU's are ill-suited 
to the outcomes movement, and one can expect them to react quite hostilely to 
any attempt by a state board or accrediting agency to measure their worth on a 
simple-minded, one-dimensional scale. Teaching, after all, is only one of the 
things faculty do, only one of the activities into which institutions invest 
their energy and resources. It seems patently unfair to make "what students 
learn" the sole basis for judging institutional quality and effectiveness. 

The dilemma in which, particularly, SU's find themselves may end in 
several different ways. As many administrators hope, the outcomes movement 
may simply go away, leaving SU's to argue over quality using all the 
traditional measures they have always used. On the other hand, the proponents 
of measuring student outcomes may succeed in convincing state legislators to 
grade— and fund— the institutions they control solely on the basis of results, 
hopefully thereby forcing SU's to redesign themselves into efficient factories 
capable of turning out the desired products—educated students. In all 
probability, however, neither extreme will be successful; the future will be a 
compromise, in which "outcomes measurement" (substantially altered in form by 
institutional reluctance) becomes an important factor in the financial formula 
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by which SU's are supported. The specific form of this compromi se^-and 
whether under it tl^e original intentions of those supporting the outcomes 
movement are preserved— wi 1 1 be shaped both Ly the means of measurement used 
and by the nature of the institutions' resistance. If measuring outcomes is 
to lead to real improvements in institutional effectiveness, the nature of the 
SU*s resistance had better be fully understood. 

The University of Louisville, a state-supported university with 
approximately 20,000 students and 1,000 faculty, is typical of SU*s torn 
between these forces. Louisville is undergoing review for reaffirmation of 
its institutional accreditation by the Southern Association of Colleges and 
Schools, a leader in the effort to make institutional effectiveness the major 
consideration in accreditation: and Louisville is under close scrutiny by the 
Kentucky Council on Higher Education, the state coordinating board for SU's in 
the Commonwealth, which--! ike state boards In neighboring Tennessee and 
elsewhere—is vitally Interested in the question of what the taxpayers get for 
the money they invest in higher education. An examination of the reasons 
behind Louisville's reluctance to embrace the "outcomes movement" will 
undoubtedly shed light on the way other SU's will react to similar situations, 
and may help the proponents of the movement better plan their strategies for 
the future. 

First, a few facts about the University of Louisville. "U of L" claims 
1798 as the date of its founding, although it did not begin operations until 
1834. Until 1972 it was a private university, supported In part by the City 
of Louisville; in 1972 financial difficulties forced It to join the stale 
system (which now has eight SU's). A condition for state support was a 
lowering of admissions requirements— from those of a fairly selective private 
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university to the "open admissions" then mandated by the state— and a lowering 
of tuition rates to statewide standards; enrollment more than doubled in the 
decade following. "U of L" consists of eleven schools and colleges (including 
Medicine, Law, Dentistry, Engineering, and Business), offers masters degrees 
in 67 fields and doctorates in 20, and supports a basketball program that has 
won the NCAA Division I national championship twice in the last decade. 
Although the undergraduate student population Is undistinguished (over 80% are 
graduates of Jefferson County, Kentucky, high school:^, and the average ACT 
Composite score for freshmen is below 18), the University has high 
aspirations: the former president's goal was to make it "Harvard on the Ohio" 
and the current (since 1981) president's goal is to make it "one of the best 
urban universities in the nation." 

Over the past three years, Louisville has mounted a major effort in 
"strategic planning," with consultative help from Bob Shirley (of NCHEMS) and 
George Keller (whose book. Academic Strategy , was required reading for all 
administrators). A strategic plan, based largely on a survey of 
"environmental opportunities" (and very little on a hard, critical analysis of 
"strengths and weaknesses") was published in early 198fi. but the funding 
required to accomplish the plan is, under foreseeable conditions, unlikely to 
be provided by the state. Briefly, the plan calls for the addition of a 
number of doctoral prograns, for the development of Louisville into a 
"research" university, for closer ties with the business community, and for 
recognition of the institution's vital importance to economic development in 
the City and state (largely through the efforts of its schools of medicine, 
engineering, and business). 
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Two other distinctive features of the University of Louisville should be 
noted. First, It has a unique arrangement concerning Its teaching hospital, 
which, although built with state funds, Is run (under a long-term lease) by a 
private hospl tal -management corporation, Humana. Second, under the leadership 
of the current president, Donald Swain, Louisville embarked three years ago on 
an ambitious fund-raising "Quest for Excellence" with a $40 million target; to 
date, over $42 million has been pledged, testimony to the strong support the 
school has locally In an otherwise rather poor state. In short, its ambitions 
for national recognition and prominence, though perhaps unrealistic, are not 
totally unfounded. It has come a long way under its current leadership. 

like Louisville, many SU's want to excel—to stand out among their peers 
as unique, as leaders. Like individuals, institutions have needs: a scale, 
like the one devised by Abraham Maslow in 1943 to categorize the hierarchy of 
individual needs, can be useful in ^athoming the stage of development reached 
by an institution. Once a SU has passed the level of concern for its simple 
continued existence (in Maslow's terms, the satisfaction of physiological and 
safety or security needs), its attention sMfts to loftier aspirations: love, 
esteem, and— perhaps— self-actual izatlon. The desire to fulfill these needs 
can be a powerful force In explaining the TOtlvatlon of behavior within an 
organization. In understanding why It ad )pts the policies It does (or doesn't) 
and why it pours Its energies Into the directions It chooses. 

Normally, Institutions seek to move up the hierarchy. Having achieved 
stable and adequate enrollments and buugets, a situation In which the majority 
of state-supported institutions find themselves, they turn their attention to 
excellence: what can we do to achieve distinction among our peers? 
Occasionally a crisi s— either In budget or enrol Iment— can momentarily shift 
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attention back to low-levei concerns, but most institutions continue, even 
through fiscal crises, to strive for ir.tangi bles— the admiration and respect 
of others. 

As mentioned earlier, the University of Louisville, like hundreds of other 
stite schools of its type, has such a lofty goal: "to be one of the best 
urban universities in the Nation." (Note that, not being fools, Louisville 
has not written a time-line into this goal statement.) That question that 
needs to be answered for such schools is "how do we achieve di sti nction?"~how 
do we go from where we are now to a position of national prominence and 
respect, and how do we do it witiiin the serious constraints imposed by finite 
limits on federal and state support for higher education; disagreements among 
faculty, students, administrators, legislators, and the public as to the proper 
function and focus of higher educational institutions; and growing public and 
professional unease with the directions higher education has taken in recent 
years. How the question gets answered is determined, in part, by the position 
legislators and the public take on questions uf irisll tutiOi":al accountaui 1 i ty; 
in turn, how state universities are held accountable is affected oy the way in 
which state universities set their goals and directions— how they decide to 
strive for excel lence. 

For state universities, adopting a vague coal— to excel— may have serious 
and problematic ramifications. Most state-supported schools belong to groups, 
and are directed (or nudged) by state regulatory or coordinating boards; they 
are not free, entirely, to decide their individual lUtures and missions. 
Although it is difficult for any state board to reject a single institution's 
desire "to excel," a state may have to make tough decisions on the parameters 
within which each institution it supports wishes to shine. 
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Kentucky, for example, presently supports two state medical schools, two 
state schools of dentistry, three state schools of law, and two state schools 
of engineering. In other states (Ohio's med^':al school situation comes to 
mind quickly), proliferation of competing efrorts—or out-and out 
duplication—may be even more obvious. Hov much of a comparatively poor 
state's revenu&s can be dedicated to the u iderstandable Impulses of each of 
its Institutions to be "stars" in the hio ily competitive firmament of higher 
education? Can a state whose Income der ves primarily from coal, liquor, 
tobacco, and manufacturing— not booming economies, at best— pour in sufficient 
resources to support three law schools, a ch of which wants to achieve 
national renown? The answer should be o'^vious. Kentucky has, among the fifty 
states, with the exception of Mi ssi s:»1 pf 1 , the smallest proportion of citizens 
who are high school graduates. Its college-going rate is one of the lowest in 
the nation; its high-school dropout ra e is alarmingly high (only two-thirds 
of students entering the ninth grade raduate four years later). [CHE 
Strategic Planning paper, T.L. GrissfTi, & R.A. Mann, "Educational Attainment 
and Careers: Overview and Findings' pp. 4-5.] Low educational attainment is, 
however, only part of the problem. Though among the contiguous forty-eight 
states, Kentucky ranks 47th in the per capita production of associate degrees, 
44th in the production of baccalcu ^eates, and 40th in the production of 
doctorates, it ranks 7th in the p eduction of dental and 15th in the 
production of medical degrees. same source] Yet 46.9% of Kentuckians 25 and 
over have not graduated from hi ^h school (compared to 33.5% for the nation as 
a who1e)~the lowest proportij of high school graduates of any state in the 
nation. [1980 U.S. Census] 
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The Kentucky CHE recently went through Us own process of "strategic 
planning" for higher education in Kentucky. In reviewing state higher 
educational priorities and budgetary constraints, Thomas L. Grissom, Deputy 
Executive Director of the CHE for Planning and Policy, concluded: 

(1) that "based on the present level of state appropriation for higher 
education in Kentucky and given the state's financial condition, the 
percentage of total state appropriation going to higher education does not 
need to be increased." [Grissom, "Budgetary Consideration and Financial 
Accountability: an Overview," 1985, p. 4.] This conclusion was, 
naturally enough, hotly opposed by the state schools' presidents, who felt 
the conclusion that "Kentucky supports higher education adequately" was 
grossly in error; 

(2) that "the present system of institutions, programs, and enrollment 
distribution is a high-cost system" that cannot be adequately supported by 
"the present tax structure or revenue allocation system of the 
Commonwealth"; and 

(3) that "Undergraduate instruction, basic research, and educational 
attainment are being sacrificed by a stubborn reluctance to reduce or 
eliminate professional educational and service programs." In effect, the 
state's schools cannot ajj strive for excellence in aH areas — 
particularly in professional education. 

Other states undoubtedly find themselves in similar situations, either 
because limited revenues cannot support a system that tries to be all things 
to all people, or because over-emphasis on graduate and professional education 
has shifted the focus of attention and interest within their systems away from 
the mainstay of higher education, the education of undergraduates. Unlike 

er|c -^^ 



SPANGEHL - 10 



many ^tates, Kentucky's loose "coordlratlng" board currently lacks the 
authcrlty to compel Institutions to cevote their resources to the things that 
the state finds Important (i.e., the education of undergraduates, and, thereby, 
the raising of educational attainment for the citizens of the state): "The 
Council [on Higher Education] does not have authority to manage Internal 
institutional operating budgets. Uniform financial renorting of fiscal year 
expenditures is required by statute but not yet fully realized. Thus, 
expenditure data are 'lOt always timely, comparable, or sufficiently detailed. 
For example, the analysis of Institutional expenditures on Intercollegiate 
athletics prepared for the Council this year required a series of specific 
requests , for data to the Institutions," [page 2, Grissom, "Budgetary..."]. 
Furthermore, "Despite statutory authority for the Council to require and 
collect all reports necessary for the coordination of the system, these 
efforts will likely face broad Institutional resistance. The claim will be 
made that the Information has no utility except for the purpose of budget 
management and control, an authority reserved exclusively to the 
institutions," [p. 3. GrIssom. "Budgetary..."]. In effect, the Kentucky 
Council on Higher Education sees the need for state\1de direction of Its 
system to focus the Individual Institutional missi ns of the state's eight 
state universities, but lacks the legislative au crity needed to hold the 
schools accountable for their decisions and act^ ns. In other states (and, I 
suspect, soon In Kentucky) state boards do poss is such power, and will, more 
and more, decide those areas upon which each rstltutlon will focus its 
efforts . 

National reports second the concerns highlighted In Kentucky's efforts to 
come to terms with Its own system. The Mortimer report. Involvement In 
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Learning (1984), and a host of successors have pointed out the tendency of 
Institutions and their faculties toward policies and practices which 
de-emphasize the Importance of basic undergraduate education In order to pay 
attention to those areas of potential "excellence" chosen by each Institution 
as Its goals. The problem here Is not In wanting to excel--one can hardly 
fault any educational Institution for wanting to be excellent. Excellence, 
however. Is a murky concept, whose Interpretation changes according to 
context. In large organizations (and state "systems" of higher education 
qualify as large organizations) excellence In a sub-unit can be at the cost of 
excellence somewhere else In the organization as a whole; every choice has 
consequences, not all of them desirable. Including the decision to strive for 
excellence. In cybernetic terms, there Is need for constant vigilance against 
suboptlmal Izatlon, the tendency of organizational units to achieve their goals 
at the expense of more Important overall organizational goals. In a 
university, for example, one can do a number of things to run a library 
efficiently while at the same time destroying the very reasons why 
universities have libraries. Policies which restrict the circulation of books 
(to cut down on loss through stealing or carelessness), acquisition policies 
designed to maximize the number of Items in a collection (by buying, for 
example, cheap governni^nt documents or microform materials that are little 
used), purchase policies which prefer the more costly acquisitions (since 
there is less staffwork consumed In buying, cataloging, and shelving one $90 
volume than ten $9 volumes) may all make a library "look" good, but may work 
strongly against the reasons for having a library in the first place. 
Suboptlmal Izatlon can occur within a state university, when the individual 
schools, colleges, and departments set their own (individually admirable) 
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goals; and within a state system, when each state university decides for 
Itself how It win excel. The Imposition of a single yi.ostkk for measuring 
results— such as the notion that state universities should be judged solely on 
how well they educate students—makes such suboptimal ization Intolerable. 

There is growing awareness that the measures used to identify excellence 
in educatiop'il institutions can easily be perverted. Institutional 
accreditation, for example, has for too long been narrowly overconcerned with 
educational resources through a simplistic assumption that the presence of 
resources automatically implied educational sufficiency and that a surplus of 
resources correlated with excellence. One need only scan the standards for 
accreditation of both regional and professional accrediting agencies to 
discern the overemphasis placed or facilities, volumes in the library, 
proportion of faculty with advanced degrees, teaching loads and 
student-faculty ratios, and the like. This fascination with the surface of 
educaticn— "surface" in the sense of its most superficial aspects— is, 
however, declining; among others, the Southern Association of Co'.leges and 
Schools (SACS) has rewritten its Criteria for Accreditation (1986) to 
de-emphasize the counting of material resources as a means of determining 
whether or not an institution offers a quality education, to shift attention 
from the number of volumes in the library to the more important question of 
whether students ever visit the library and read the books it has. This shift 
has been slow in coming, perhaps for good reasons: during the explosive 
expansion of the American Higher education system during the sixties and 
seventies, accrediting agencies spent their energies simply in keeping up with 
the growth of new institutions and the expansion of existing ones (all those 
community and state teachers colleges which suddenly became universities with 
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doctoral programs); a superficial overview of whether new or expanded 
Institutions had the minimal resources necessary to their intended missions 
seemed to be enough. 

Now, with demographic projections which Indicate that the U.S. probably 
has more places for students In higher education tlian It needs (and certainly 
has more places In graduate and law schools for students than society needs 
Ph.D.'s and lawyers— although lawyers, unlike Ph.D.'s, seem to be consistently 
able to create new demand for their services), the concern Is shifting from 
adequacy to quality, from looking at the superficial resources to the 
processes and products of our higher education system. Folklore (which may. 
In fact, be true) has It that SACS' change In emphasis was motivated by a 
threat from (of all places) Florida to withdraw from the SACS Commission which 
accredits secondary schools In me south: the Southern Association had 
threatened with loss of accreditation one of Florida's best high schools (best 
as judged by the proportion of stuoents going on to college) because It was 
located too close to a railroad track. Supposedly, Florida threatened to 
leave the association unless quality were taken seriously In accreditation 
matters; as a consequence, SACS rewrote Its criteria for granting and renewing 
accreditation. SACS Is a leader In this area, but It Is not alone; the 
criteria used to judge higher education are changing quickly. 

Often, the form In which a question Is asked determines Its answer, a 
truism which holds tor the question "what Is excellence?" In higher 
education. To foresee how Institutions will react to this growing tide of 
sentiment in favor of tightly focusing Individual Institutional missions and 
of measuring student learning as an Index of educational excellence, it 1s 
useful to rephrase the question "what 1s excellence?" into a form that 



SPANGEHL - 14 



reflects the perspective of an individual college or university: what can we, 
as an Institution, (or I, as a president or chief academic officer) do to make 
my Institution stand out, to make It receive the admiration, respect, or envy 
of others? Phrased this way, the question often invites answers very 
different from those offered by the authors of reports and essays that suggest 
outcomes assessment as a way out of the "crisis" in current American higher 
education. Moreover, the answers to this question that occur to 
administrators In state-supported institutions fall into a narrower range than 
those available to private schools, 

Ther« are traditionally ilve or six different avenues to excellence — to 
making a college or university known and respected by others. These include: 
(1) amassing resources (people, buildings, equipment, endowments, etc.) so 
that the Institution Is materially superior to Its peers; (2) restricting the 
quality of "Inputs" (I.e., students admitted) to a high level; (3) acquiring 
prestige on the basis of outstanding accomplishment in a relatively narrow 
area of enterprise, either academic (e.g., a strong honors program, a 
respected medical school) or non-academic (e.g., a winning athletic program); 
(4) attracting and nurturing a prolific faculty who can win fame for the 
school through their scholarship and research; (5) graduating only those 
students who will achieve a record of distinction after college; or (6) 
incorporating into the structure of the educational process some distinctive 
element which makes comparison with other institutions difficult (e.g., 
individualized self-directed study programs, liKe those at Empire State 
College in New York). Not all of these avenues are equally available to al^ 
schools, even fewer to state universities. 
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First, to excel In higher education traditionally has meant to have 
greater resources than others, to succeed at the very things measured by the 
sccrecards accrediting agencies traditionally use. Size of endowments; square 
footage 'devoted to classrooms, offices, and laboratories; number of doctoral 
faculty; low faculty-student ratios; size of library collections; and the like 
are all objective facts that enable schools to compare themselves to others. 
The difficulty with striving for excellence by amasFing material wealth 
(money, manpower, machines, etc.) is that, for a state university, the contest 
is already too competitive. State schools are not only in competition with 
each other, but with all the other agencies supported out of a state's 
budget: prisons, health care, welfare, roads—many of which are perceived by 
the public (and, consequently by legislators) to be of more immediate import. 
It is difficul t— probably impossible today— for a state university to argue 
for an increase in its appropriation when the alternative uses for the money 
are so obvious: improving mass transit, keeping criminals off the streets, 
stopping the drug trade, insuring a decent life for the elderly, etc. Add to 
this competition for funds the worn approach that higher education uses— "just 
give us the funds, and we'll use them wisely to improve things"— and you have 
a notably unconvincing argument for most legislators. Thus the institutional 
impulse to seek distinction through increased resources has, more and more, 
been stymied or redirected. Although state universities still lobby for 
bigger state appropriations and state approval of tuition increases, they have 
also turned their attention to fund-raising for affiliated foundations (where 
permitted) and to increasing their share of a shrinking pot of federal and 
private grant and contract money. For evidence of this, one need only look at 
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The Chronicle of Higher Education , particularly at turnover in and competition 
for "Institutional development" officers. 

Louisville's successful "Quest for Excellence" has produced nearly $38 
million In "unrestricted" (I.e.. not accountable to the state) funds. In 
itself, private and corporate support for public higher education is not a 
problem, although It does create difficulties for state systems which have 
comprehensive strategies for how they would like to see their institutions 
develop; the privately-raised money sometimes takes institutions off in 
directions which do not fit the state's plan. The quest for grant and 
contract money, sought because the overhead allowance also provides the 
institution with a "margin for excellence" does, however, have profound 
effects on the functioning of an Institution. To push its faculty into the 
competition for research grants, many Institutions have made profound shifts 
in personnel policy: the relative attention given to teaching versus research 
has declined dramatically, and even lackluster former state teachers colleges 
which prided themselves on a long tradition of focusing attention on students 
have become "publish or perish" institutions. Moreover, there are, even at 
schools with longer traditions of that faculty research, disturbing 
trends— especial ly an increasingly obvious tendency ^o favor applied over 
"pure" research, a favoring of short over long-term research projects, and a 
blatant preference for the hard sciences and engineering (where grant and 
contract money is more abundant over the arts and humanities). One can expect 
:tate-supported schools to continue to seek resources through these devices, 
with perhaps, one additional, consideration: lobbying for state support has 
taken a new tack, with institutions appealing to legislatures for increased 
appropriation on the grounds that the presence of a university in an area, 
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particularly a university with a host of doctoral programs staffed by active 
researchers, will stimulate the area's economic development. Although the 
evidence is sketchy that business relocate and entrepreneurs spring up in 
p.reas with state universities, this argument may be convincing for a while and 
may benefit institutions (particularly those with engineering acid business 
schools), the strategy will be effective only if rather direct casual 
connections can be drawn between Increased economic activity and increased 
state support for an instUution. Given the crisis in confidence over the use 
universities have made In the past in turning their Increased appropriations 
Into better educated graduates, the chances of an Institution enriching itself 
and achieving distinction on these purely economic arguments seem slim. It 
may not hurt to try for more state money, but it is probably more realistic 
for SU*s to seek distinction. 

Another traditional measure of excellence has to do with what, for lack of 
a better term, can be called the quality of raw materials. Selectively and 
incoming student quality Is often cited as a mark of distinction; the best 
schools are the ones to which the best students come, the ones which admit 
only those who can be educatod with the least effort. For state schools, 
distinction through selective admissions is difficult: often there are quotas 
or differential tuition rates which discourage out-of-state recruiting 
efforts, and the pool from which the schools draw is rather narrow. Many 
state schools have a largely regional or local prospective student population 
and—with declining numbers of 18 years olds and falling entrance exam 
scores—the desire to be more selective must be balanced against the desire to 
continue to exist. Fearful of lowering enrollments (and laying off faculty), 
most ::hools are happy to take whomever they can get. The increased 

18 



SPANGEHL - 18 



permanence of remedial programs is testimony to the impossibility, for most 
SU's, of attaining distinction by rejecting applicants. Moreover, concerns 
about educational opportunity for minorities and federal rules concerning 
desegregation make truly selective admissions a near impossible route to 
distinction. Louisville, for example, lives under an Office of Civil Rights 
agreement to remove the vestiges of segregation in the Kentucky state system; 
its target is to graduate the same proportion of undergraduate Blacks as are 
present in the local population, no mean goal in a city where the rate of 
college-going among Blacks is on the decline and where recruiting out-of-state 
Blacks supposedly undermines the state's purpose in maintaining a higher 
educational system. Truly selective admissions is a strategy for excellence 
best left to private institutions. 

These two approaches— increase resources and limiting the pool of students 
admitted—are traditionally the marks of outstanding institutions. Astin's 
(1986) study shows clearly that the institutions most people perceive as best 
are those which have these traits. These are not, however, workable 
strategies by which SU's can hope to attain distinction. 

Prestige can, nevertheless, be earned in other ways. Sports is one: 
Louisville, winnei of the 1980 and 1986 NCAA national basketball championship, 
gained more attention through athletics than it ever garnered academically. 
Sports are popular with legislatures; money flows much more easily during the 
thrill of victory than at other times. But sports are costly; schools don't 
win championships by doing well the things for which educational institutions 
evolved, and sports sap a great deal of attention away from other areas of 
concern. Competition, of course, is intense; all Division I schools in the 
NCAA are out to be champions, and excelling in such a selective arena has a 
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high price. In the wake of Louisville's 1986 championship came three lasting 
achievements: the legislature approved the construction of a new student 
center (more properly a physical education-student center); Louisville 
committed itself to seeking national prominence In football (hiring as coach 
Howard Schnellenberger of the Miami Dolphins); and approximately one million 
dollars per year in gifts were diverted from the annual campaign used to 
support academic operations into the annual athletic fund-raising drive (whe.e 
contributions of given amounts became prerequisites for the purchar>e of season 
football and basketball tickets). Sports can make an institution famous, but 
sports In themselves can do little to make it more famous as an educational 
institution. 

Athletic prominence is just one possible area for a "pocket of 
excellence"; many others are possible, and they can be directly connected with 
a SU's educational mission. SU's with distinguished individual colleges or 
departments are not uncommon, and the institutions which have such pockets of 
excellence" ought to be proud of them. Too often, however, "pockets" do not 
seem enough—what institutions want Is an overall reputation that draws rave 
reviews, not simply an art or history program that shines. Moreover, 
conscious attempts to create individual outstanding programs often create 
internal havoc, since the piograms which have not been "targeted" for a 
starring role understandably feel neglected. Then too, it is difficult (from, 
say, a statewide perspective) to understand why a disproportionate share of an 
institution's resources should be vested in a department which benefits only a 
relatively small number of faculty and students. Legislators or accreditation 
teams may ask why the funds, if available, were not used for more fai-rearhing 
improvement of a SU's overall program. 
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Graduates and alumni, if distinguished, can do much for a school's 
reputation, but the fame and wealth of alumni is heavily dependent on the 
social and economic level of students admitted in the first place. As Astin's 
data demonstrates, student social class (measured on a variety of variables) 
is highest anx)ng those institutions with selective admissions policies. In 
the competition for distinguished alumni, the Harvards and Yales and Stanfords 
have quite a head-start on the Slippery Rocks, Akrons, and Louisvilles of the 
world. Theoretically, one might conceive of a state school so excellent at 
educating students that. In spite of students* origins and entering ACT 
scores, its graduates go on to head corporations and win fame as artists and 
writers, but the theory on which such a conception is based is risky—too 
risky for most college administrators. To rededicate and restructure an 
institution in hopes that it will win fame through its graduates is unlikely 
to become a common strategy for the nation's state colleges and universities. 

Given the difficulty of competing for prestige on these traditional 
grounds, several state institutions have attempted to find for themselves a 
distinctive niche --a category which will allow them to achieve distinction 
primarily because there are few (if any) others with whom to share it. New 
York's Empire State College, for example, is a leader in individualized, 
distance-learning programs for ^^dults — both because the College's faculty and 
administration are good at what they do and are dedicated to doing it, and 
because there are few, if any. other schools fighting for that particular 
title. Similarly, Northeast Missour, State, under the far-seeing leadership 
of Charles C. McClain, embarked on a comprehensive and rigorous program of 
measuring student outcomes as early as 1972--then, after the evidence was in, 
asked for an additional state appropriation to reward and encourage their 
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distinctive efforts. Unfortunately, strategies like these cannot work for 
every school 1n a state: if every SU in a system collects student outcome 
data and asks for appropriations based on these results, all will find 
themselves locked in competition for a single pot. To be the only school in a 
state measuring outcomes provides a certain distinctiveness; if all do it, 
resul ts are what counts (as, for example, in the limited outcomes-based 
formula-funding experiment in Tennessee). Yet, simple and direct as it 
sounds, staking the future of an institution on the hope that one's students 
will learn more than similar students at similar institutions is a risky ''nd 
dangerous venture, particularly for administrators who can envision more 
traditional (and potentially less damaging) ways of striving . distinction. 
Without outside pressure (from state boards and legislators, in particular), 
the SU that chooses to pursue fame and fortune by demonstrating that its 
students learn more will be rare indeed. 

Although measuring the effectiveness of an institution in achieving its 
primary function—educating students— seems like an uncontroversial idea, it 
is likely to face extreme resistance within institutions, particularly SU's. 
The reasons for this resistance are numerous, and differ from one campus group 
to another. Administrators, faculty, students, and alumni can all be expected 
to fight the impulse in one way or another, and the nature and degree of each 
group's recistance will, in the end, be important in determining the ultimate 
form that evaluation takes. 

From the administrative perspective, assessment has clear minuses and 
uncertain pluses. First, ifs expensive; Ewell and Jonts [The Costs of 
Assessment] figure that a minimal, basically one-dimensional assessment 
program for a mid-size school will run $125,000 per year. Costs either have 
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to be passed on to students, or funds have to be reallocated from other 
projects. If an assessment program is to be truly insti tutional~if it is to 
examine the effectiveness of not just the educational programs of an 
institution, but all of the myriad research, service, and administrative 
programs which compete with the educational programs for funds and attention 
(and upon whose effectiveness, ultimately, the educational programs 
depend)— the expense of assessment will rise meteorically; yet to measure 
carefully and hold accountable instructional programs only, while allowing all 
the other programs which eat up an institution's resources to go their own 
merry ways, seems unconscionable. A narrowly focused institutional evaluation 
effort has the potential to split (further) the faculty from the rest of an 
institution's employees, to reduce (further) faculty morale, etc. 

Imposing an evaluation program is, therefore, a dramatic and expensive 
move, one that had better hold in store clear benefits: otherwise, no 
clear-thinking president, provost, or dean would subject his or her 
institution to the ordeal. What makes It worthwhile? When do the prospective 
benefits outweigh the costs? 

Clearly, the benefits lie in improving the educational process--in using 
the results of the assessment system to make appropriate changes in 
curriculum, scheduling, personnel systems (i.e., appointment, assignment, 
development, and evaluation in reward of faculty), and in adjusting all of the 
other factors which may contribute to a better education for students. Taking 
on this challenge requires forceful leadership, in most cases at the 
presidential level (especially if the assessment program is to be 
institution-wide), yet estimates indicate that less than TL of university 
presidents are directly involved in academic affairs: and presidents have 
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more pressing matters before them than the improvement of undergraduate 
education. More importantly, perhaps, assessment systems will add a plethora 
of computer printouts to an administrative environment which is already 
drowning in data. Ai> George Keller points out in Academic Strategy , the model 
for the modern university president is the forceful decision-maker, the 
action-^oriented mover-^-not the studious seeker of additional data. Without a 
clear vision of how the data from assessment studies can be translated into 
action (a difficult translation at best in an academic tradition which 
requires an endless series of committee meetings and debates before even the 
simplest curricular change can be permitted), it is unlikely that many 
presidents will take it upon themselves to lead their campuses to a 
comprehensive system of measuring student outcomes. 

Pressure from administrators below the president might move outcomes 
assessment up in a SU's list of presidential priorities, but other problems 
interpose themselves. To advocate student outcomes as the final measure of 
quality is to admit inadequacies in the current system by which programs and 
students are certified, yet the current system is, in most cases, largely the 
creation of those charged with administrative responsibilities. To lambast 
it, to say it needs to be replaced with a new system that looks at product 
rather than process, is tc risk the ire of the faculty, who will naturally 
rile at the idea that the curriculum, programs, and teaching for which they 
are responsible could be dramatically improved by the imposition of a new 
bureaucratic creations-outcomes assessment. 

The faculty, of rourse, cannot be expected to eiibrace a bureaucratic 
procedure designed to test whether or not the curriculum (which is organized 
and delivered by the faculty) is effective: if the faculty thought it needed 
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revision or that their teaching needed improvement, they would take action. 
It might, however, be possible to alleviate this faculty fear that outcomes 
assessment is designed to "check up on whether the faculty are doing their job 
well" by Instituting an outcomes measurement system throughout an 
institution: to use measures of all institutional activities to determine the 
effectiveness with which all parts of a university contribute to the "missior " 

Faculty fears spring from other sources as well. One is the belief, 
hopefully wrong, that outcomes must be measured by paper-and-penci 1 tests. 
Although SACS and other groups favoring assessment have gone to great lengths 
to suggest alternatives and complements to testing (e.g., exit interviews, job 
placement data, alumni surveys, library usage information, etc.), plans to 
assess the outcomes of a college education, where they have been implemented, 
have depended heavily on testing—particularly on the ACT COMP battery and on 
the subject area GRE examinations. Yet few faculty would use a single test as 
the measure of student progress in a course— grades are usually based on 
multiple measures including participation, papers, and other factors. To use 
one or two te:ts to measure the collective impact of a department or an 
institution on a student seems ludicrous. When faculty discuss the issue of 
outcomes assessment, they raise a series of valid concerns, all of which will 
have to be addressed before they will accept and cooperate with a systematic 
attempt to measure educational effectiveness: the tendency of such a system 
to encourage "teaching for the test"; the tendency of a measurement system to 
encourage the setting of lower goals than wou^d otherwise be desirable; the 
need to recast educational goals into a form that will permit their 
achievement to be observed or measured; the belief that many of the "benefits" 
of an education do not become visible (much less measurable) until years or 
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decades have passed; etc. Then, too, there Is the substantial problem of what 
one does with the results of such an assessment: If the faculty of College X 
learn that their graduates are Inferior to those of College Y, how can this 
information be used to Improve College X? These are all serious concerns, and 
must be addressed If institutional effectiveness is to be measured for the 
benefit of an Institution. 

Students, too, are likely to be concerned, as will alumni. First, testing 
is time-consuming (if not expensive); few students advocate more testing. 
Second, students are concerned, rightfully, about the use to which the results 
will be put. If only outcomes are measured (i.e., if the tests are designed 
and used co assure a minimal level of quality control among graduates), their 
impact—in the long run—may be to restrict the admission of freshmen, since 
the better-prepared admlttees can be expected to outperform others, even on 
o»xit exams. On the other hand, if outcomes are measured as part of a system 
designed to determine "value-added" (i.e., the average gain In scores made by 
the students of a given Institution between admission and graduation), then 
the pressure may be In the reverse direction: a college can make itself look 
good by accepting only those with the greatest potential for Improvement. (In 
a few cases in which Institutions have experimented with the "value-added" 
approach, comparative scores show a net loss as a result of four years' of 
education; clearly, such schools would have done better had they started with 
more Ignorant students.) If the results of assessment are made public— and 
one can hardly expect a school with documented evidence that It provides a 
better education than Irs competitor: to keep a secret— these concerns will be 
aggravated. How would It feel, for example, to be the alumnus of an 
institution which advertises that It now provides a vastly better education 
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than the one you received? Kentucky now formally prohibits Its SU*s from 
publishing claims that one offers a better e'jcatlon than another, but, If the 
evidence were available, could such Information be supressed? If it were not 
supressed, what would the effects be on a state system? Plunging blindly Into 
outcomes assessment when so many important questions remain unanswered is 
extremely dangerous. 

In spite of all the resistance with which the novement will be met. It is 
clearly a force that will continue to exfirt pressure on higher education for 
some time to come. Legislators, unable ^-o see the subtle problems with 
outcomes assessment that are so obvious to faculty, will press to see the 
"bottom line," and accrediting agencies (through COPA's Influence and SACS' 
example, among others) will undoubtedly continue to Insist that institutions 
demonstrate their success in achieving the goals they have set for 
themselves. If chaos Is to be prevented, and if all the energy which will be 
Invested In the attempt to measure effectiveness is to come to some good, 
compromises will have to be struck, and the objections of administrators, 
faculty, students, and alumni will have to be reckoned with. The following 
are some suggestions (from Louisville's experience and from the growing 
literature on the subject) that may help narrow the gap between those 
advocating the assessment of student outcomes and those reluctant to adopt the 
notion: 

(1) Astin's term, "talent development," is preferable to the 
alternatives. "Value-added," with Its economic metaphor, strikes faculty 
as natu* lly offensive; "outcomes" seems to Ignore process and focus 
solely on results. In addition, "talent development" can apply both to 
students and faculty; the goal of an institution can be seen as the 
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development of both groups' talents. Given that many institutions place 
great stress on faculty development (especially the development of active 
researchers and publishers), Astin's term provides room for existing 
Institutional goals to be included. Its wider use (and adoption 01' the 
concept on which it is based) will go far in reducing faculty hostility to 
the notion of assessment. 

(2) At large, complex institutions, common goals cannot be assumed. Yet 
the construction of an assessment system, in which all students are 
measured to determine the degree to which they have achieved common goals, 
demands this assumption. To establish an outcomes measurement system, one 
must either define, ex cathedra, the goals of an institution, or be 
willing to experiment with possible goals and possible ways of measuring 
their achievement. To succeed at implementing a system, administrators 
would do well to follow the latter course, to let the common goals and the 
instruments to measure them evolve through a long period of discussion and 
experimentation^-perhaps several years. 

(3) Successful measurement requires an atmosphere of openness and 
Informality. Faculty need time and freedom to try out potential goals and 
measures; if they are held accountable for their "results" too quickly, 
they v-ill inevitably settle for the most easily attainable goals and the 
least useful measures. 

(4) Multiple measures of student "education" must be used; a system 
cannot depend on one or two paper-and-penci 1 tests. The treasures must be 
appropriate for each program as well. Using a variety of .neasures. ones 
which must often be custom-designed for specific purposes, 15 both 
expensive and time-consuming, yet necessary. 
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(5) Professional schools, which have had long experience with 
certification and board exams, may see the issue very narrowly; the 
impulse to define a "well-educated" dentist as one who passes the Dental 
Boards must be resisted at all costs. 

(6) Focusing on student outcomes alone may create serious conflicts with 
the other goals to which an institution wishes to devote its energies: 
research, public service, etc. Most SU's claim that the "transmission of 
knowledge" is only one of their goals; an outcomes measurement system 
should also record success in the creation of new knowledge (i.e., 
research) and the application of knowledge to specific problems (i.e., 
^.ervlce) . 

<7) The question of "what will we do with the results?" must be faced in 
advance. Institutions which have experienced the greatest success with 
outcomes assessment are those in which the concern with effective 
education preceded the imposition of a measurement system—where the 
faculty adopted an evaluation scheme because they seriously wanted to know 
what they were doing well or poorly. In such a setting, the results of 
evaluation are Ihf- answer to a question already asked. A real concern for 
improving education must exist before a measurement system is imposed. 
(8) The entire process of developing an assessment system must be done in 
a cMmate of openness and trust—administrators, faculty, and students 
should know and discuss with each other the purpose of the system and the 
intended use of the results before the system is implemented. 
(9, Outcomes can be measured in all of the activities of an institution; 
measurement should not be limited solely to educational activities. 
Limiting the focus sends the wrong message to faculty: if there are 
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Inefficiencies in this institution, you are responsible. A more 
comprehensive system of measurement, in which the effectiveness of the 
payroll department, the student housing department, and the English 
department are ali measured and Improved, will be good both for faculty 
and for the institution. 
In conclusion, measuring outcomes is a risky path for an institution, and must 
be approached with care and planning. Improving education—turning out more 
educated students— is unlikely to be the road to fame for any institution; 
improvements are gradual, difficult to perceive, and hard to communicate to 
others. Many, perhaps most, schools are likely to continue to search for 
other means of achieving distinction. Yet, at base, education is the reason 
colleges and universities exist, and— particularly for SU's— those who support 
higher education are certain to continue to demand hard evidence of the 
success institutions have in doing the job they are funded to do. Resisting 
the assessment tide Is foolish; wise institutions will seize the initiative, 
and begin the long task of developing ways of measuring their effective- 
ness—both to satisfy their state boards, accrediting agencies, and publics, 
and to help them become better at their primary function. 
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